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(54) Title: INTEGRATED TELECOMMUNICATION COLLABORATION SYSTEM 



(57) Abstract 

An integrated telecommunication collaboration systetn 
which allows the close integration of desktop computer appli- 
cations, data communications, and public switched telephone 
network voice connections. Client software which implements 
the invention includes a voice agent (304) for establishing tele- 
phone sessions, a data agent (305) for establishing associated 
data sessions, a communications agent (303) which synchro- 
nizes the date and voice agents, and an integration agent (302) 
which integrates the data, and voice communications with a 
computer operating system, and with various applications. The 
integrated telecommunication collaboration system controls the 
establishment of harmonized calls by sending and receiving 
various requests and status messages to and from a server (704, 
708), and another client. 
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5 INTEGRATED TELECOMMUNICATION COLLA BORATION SYSTEftft 

TECHNICAL FIELD 

This invention is related to tiie field of integrated desktop telecommu- 
nications. In particular, this invention relates to a way in which applications, 
10 data communications, and public switched telephone network (PSTN) voice 
calls can all be integrated together in a desktop computer system. 

BACKGROUND 

Presently, voice telecommunications and data telecommunications 
15 are largely separate to most computer users. A desktop computer user can 
run an application which includes integrated multimedia communications, 
however, if the user wishes to initiate a voice communication, the user must 
use a separate PSTN telephone line. Alternatively, the user can make use 
of a data communications system which includes voice capability, where the 

20 voice communications path is terminated by the user's personal computer. 
Such a system however, makes use of some type of voice-data conversion, 
such as voice-over-Internet-protocol, resulting in lower quality voice commu- 
nications than is typical of the PSTN. Additionally, typical multimedia tele- 
communication applications do not allow for close integration with other 

25 desktop applications such as office suites, presentation software, word 
processing software and the like. 
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5 A typical multimedia telecommunications platform is specified in 

H.323, a well-known standard of the International Telecommunications 
Union (ITU). Application programs such as Microsoft Netmeeting™ allow 
the integration of voice communications together with multimedia telecom- 
munications, however, the quality of the voice connection is poor. A high- 
10 quality voice connection can be made through the PSTN, however, this 
connection must usually be manually initiated. Additionally such a connec- 
tion does not allow for close integration of the voice communications with 
the multimedia data communications. Products such as Voice Button™, 
marketed by the assignee of the present application, allow a PSTN call to be 
15 initiated from the computer user interface, however the call is not closely 
integrated with any sort of data communications system and does not allow 
application sharing. Additionally, such a call cannot be initiated from within 
desktop computer application suites. A description of a way a phone call 
can be initiated from the computer desktop can be found in U.S. patent 
20 application number 08/948.975. entitled, " Method and Apparatus for Origi- 
nating Voice Calls from a Data Network." Filed October 10. 1997. which is 
assigned to the assignee of the present application and incorporated herein 
by reference. 

What is needed is a way to provide integration between data and 
25 high-quality PSTN voice. Such a system should not only result in automated 
PSTN voice call origination through the public network, but also provide for 
the integration of voice and data communication components into common 
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5 desktop computer application suites. The system should allow users to 
initiate a call having both voice and data components from \N\Xh\n typical 
desktop applications and from within a generic operating system user inter- 
face. The system should also not require any specialized customer prem- 
ises equipment (CPE). 

10 

SUMMARY 

The present invention solves the above problems by providing a 
framework: for the continuous delivery of value added communication serv- 
ices. The present invention enhances overall human communication effi- 
15 ciency by extending public switched telephone network (PSTN) voice calls 
with data and application sharing capabilities. The present invention con- 
sists of client software which integrates seamlessly with a personal, desktop 
computer operating system and desktop computer applications. The soft- 
ware controls communications through the PSTN, and through the public 
20 data network, and with any necessary servers within either of the networks. 
We call the invention an Integrated Telecommunications Collaboration 
System or an "ITCS". We further refer to a call initiated and maintained by 
such a system as a "harmonized" call, because the voice session, the data 
session, and one or more desktop applications are working in harmony. 
25 Wfth the present invention, a user need only initiate one harmonized call 
which results in a PSTN voice session originating through the PSTN, and an 
associated data session established over a public data network connection. 
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5 The present invention can request a server such as a computer 

telephony server to establish a telephone connection with the calling user. 
In effect, the user's computer rings the user's telephone. Upon receiving a 
notification that the connection is established, the client software of the 
present invention requests the server to offer a harmonized call to a called 

10 user. When the client receives a notification that the harmonized call has 
been accepted by the called user, the client establishes the telephone 
session and associated data session with the called user. In an alternative 
embodiment the calling user initiates the harmonized call by picking up the 
telephone receiver. This results in an off-hook notification to the client 

15 software, allowing the client software to initiate a harmonized call. Option- 
ally, the client software can notify the user of the progress of connecting the 
call. When the call is complete, the client receives a release notification 
message, and releases the telephone connection and the associated data 
connection. 

20 The client software of the integrated telecommunication collaboration 

system can manage calls either with another integrated telecommunication 
collaboration system or with other types of data communication clients, 
which support the same data communications standards as used by the 
ITCS. When an integrated telecommunication collaboration system accord- 

25 ing to the present invention receives a call offered notification message, it 
requests the server to establish a telephone connection with the called user. 
Again, the client software rings the user's telephone. After receiving a 
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5 notification that the telephone connection with the called user has been 
established, the client software participates in establishing a telephone 
session and the associated data session. 

The integrated telecomnnunication collaboration systenn client ac- 
cording to the present invention includes a voice agent for establishing 

10 telephone connections through server, and the data agent for establishing 
associated data connections with other clients. A communications agent 
connected to both the voice agent and the data agent observes the voice 
agent and the data agent, synchronizes the. communications of the voice 
agent in the data agent, and provides interfaces for plurality of service 

15 agents. An integration agent is connected to the communications agent and 
processes user input, makes requests, and provides interfaces to applica- 
tions. 

The software which implements many aspects of the present inven- 
tion can be stored on a media. The media can be magnetic such as disk- 

20 ette, tape or fixed disk, or optical such as a CD-ROM. Additionally, the 
software can be supplied via the Internet or some type of private data 
network. A workstation which typically runs the client software includes a 
plurality of input/output devices a connection for a network and a system unit 
which includes both hardware and software necessary to run the Integrated 

25 Telecommunications Collaboration System client. A client workstation 
according to the present invention typically operates in a network which 
includes a PSTN, a public data network which has a service provider point 
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5 of presence for providing the public data network connection, and a com- 
puter telephony server which is connected to both the PSTN in the public 
data network. The user's telephone is connected to the PSTN and the 
Integrated Telecomnnunication Collaboration System is connected to both 
the PSTN and public data network. 

10 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a call flow diagram which illustrates how a harmonized call is 
initiated to a called user who is using the Integrated Telecommunication 
15 Collaboration System of the present invention. 

FIG. 2 is a call flow diagram which illustrates how harmonized call is initiated 
to a called user who is not using the Integrated Telecommunications Col- 
laboration System of the present invention, 

20 

FIG. 3 is a functional block diagram showing the interconnection of the 
various software components of the Integrated Telecommunication Collabo- 
ration System of the present invention. 

25 FIG. 4 is a message flow diagram which illustrates the communication 
between the various components of the software client of the present inven- 
tion when a call is initiated from a computer desktop. 
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5 

FIG. 5 is a message flow diagram which illustrates the communication 
between the various components of the software client of the present inven- 
tion when a call is initiated by lifting a telephone receiver. 

10 FIG. 6 is a message flow diagram which illustrates the communication 
between the various components of the software client of the present inven- 
tion when a harmonized call is received. 

FIG. 7 shows the network environment in which the present invention is 
15 used. 

FIG. 8 illustrates a client workstation implementing the present invention. 

FIG. 9 shows one example of a medium on which a computer program 
20 which implements the present invention may be stored. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

To illustrate how the invention operates, we will first describe the 
high-level call flows between the various components in a network where 
25 the invention is in use. As previously stated, we call our invention an Inte- 
grated Telecommunication Collaboration System, for which we will use the 
acronym "ITCS" for the sake of brevity. Throughout the following discussion, 



wo 99/55099 



PCT/US99/08236 



5 we refer to the calling user as "Userl" and we refer to the called user as 
"User2". When we refer to an ITCS client, we are referring to client software 
running in a user's personal computer or workstation. When we refer to a 
desktop computer, we are referring to a computer which can sit on a desk- 
top such as a personal computer. Although, such a system can sit on a 

10 desktop, it can also be a mobile or laptop computer, and such terminology is 
not meant to limit the operating environment of the invention. When we 
refer to a computer desktop, we are referring to the "desktop" interface of a 
graphical user interface based operating system, 

A data communications platform forms a part of the invention. A 

15 platform with multimedia capabilities provides the most functionality to the 
user. In the preferred embodiment, the ITU H.323 platform is used and 
therefore H.323 is referred to in this discussion by way of example. As 
described below, it is possible for an ITCS to initiate harmonized calls to a 
non-ITCS client. As an example, we use a generic H.323 compliant client, 

20 that is an H.323 client without ITCS implemented. When we refer to a User2 
H.323 client, we mean a generic client without ITCS implemented. 

ITCS User to ITCS User Call Flows 

Figure 1 describes the sequence of messages sent between the two 
25 user's clients and telephones and the computer telephony server 
fCTSen/er") during the establishment and setup of a harmonized call 
between two ITCS users. The CTSen/er may be a "stand-alone" CTServer, 
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5 a TAPI enabled phone device which supports similar requests and notifica- 
tions, or a PSTN switch with built-in CTServer capability. It is also possible 
to provide the function through the advanced intelligent network (AIN), which 
is well-known. First. User1 notifies its client that the user wishes to originate 
a harmonized call. As will be described in more detail later, this notification 
10 can either be by some interaction with the computer through the user inter- 
face, or by picking up the receiver of the telephone. The User1 ITCS client 
requests the CTServer to establish a PSTN connection from the CTServer to 
UserVs associated phone. This request is accomplished by sending a 
Receive Notification Request 101 to the CTServer. The CTServer will 
15 originate a PSTN Setup ring signal 102 to Userl's phone causing the phone 
to ring. Once the CTServer detects an Answer 103 from the phone, the 
CTServer sends an Off-hook Notify message 104 back to the associated 
ITCS client. The client responds with an Offer Call Request message 105 
back to the CTServer supplying the Directory Number (DN) of User2. 
20 Upon receiving the Offer Call Request message from the User1 Cli- 

ent, the CTServer will notify the User2 Client associated with the user being 
called by sending the User2 client a Call Offered Notification message 106. 
The User2 Client responds to the Call Offered message by sending a Re- 
ceive Notification Request message 107 back to the CTServer requesting 
25 the server to ring the phone associated with the User2 Client. The 
CTSen/er originates a Setup ring signal 108 to the User2 phone. Upon 
receiving an Alerting message 109 back from the phone, the CTServer can 
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5 notify the User1 Client that the called phone is ringing by sending it a Ring 
Notification message 110. 

When the called user (User2) answers the phone, the CTServer will 
receive an Answer nnessage 111 and notify the User2 Client by sending the 
User2 client an Off-Hook Notification message 112. The User2 Client 

10 responds to the Off-Hook Notification message by sending an Accept Call 
Request message 113 back to the CTServer. When the CTServer receives 
the Accept Call message from the User2 ITCS Client, the CTS notifies the 
calling user that the voice connection has been accepted by sending the 
Userl Client a Call Accepted Notification message 114. Now the PSTN 

15 voice session is established between the Userl phone and the User2 
phone. 

The Userl Client now proceeds to automatically establish an H.323 
data connection to the User2 Client. First. Userl Client initjates a search 
request 115 to perform a lightweight directory access protocol (LDAP) 

20 lookup. LDAP is a standard used in the Internet for directory queries. 
These directories can be public or private. The query intiated at 115 is 
performed on a set of network centric directories using the e-mail address of 
User 2 as the distinguished name in the lookup. The LDAP directory server 
responds to the User 1 client with a Search Response message 116 identi- 

25 fying the current data network address of the User2 Client. In the preferred 
embodiment, the data network employed will be a TCP/IP network and the 
address will be an IP address. Upon receiving the LDAP Search Response 
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5 message, the User1 Client sends an H.323 call Setup message 1 17 directly 
to the User2 Client requesting to establish an H.323 data call. As soon as 
the User2 ITCS Client receives the Setup message, it immediately responds 
with an H.323 Connect message 118 since the User2 user has already 
indicated a desire to communicate by just having answered the phone. A 
10 harmonized call has just been established resulting in a PSTN voice session 
between both users as well as an associated H.323 compliant data session 
between the two ITCS clients. 

When Userl hangs up the phone, a Release message 119 is sent to 
the CTServer. The CTServer propagates the Release to the User2 phone 
15 as Release message 120 resulting in the complete release of the PSTN 
voice call. The CTServer then proceeds to notify both the Userl and User2 
Clients that the voice session has been released by sending a Release 
Notification messages 121 and 122. Once the Userl Client receives the 
Release Notification message 122 from the CTServer, the Userl Client 
20 immediately takes down the H.323 call by sending an H.323 Release mes- 
sage 123 to User2. 

ITCS User to Non-ITCS User Call Flows 

Figure 2 describes the sequence of messages sent between two 
25 user's clients and telephones during the establishment and setup of a 
harmonized call between an ITCS client and a non-ITCS client. The user 
who does not have an ITCS does have a generic H.323 compliant client 

-11- 
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5 running on his/her desktop computer. Userl originates the call in an identi- 
cal fashion as shown in Figure 1. Messages 201 through 205 are identical 
to messages 101 through 105 of Figure 1. 

Upon receiving the Offer Call Request message 205 from the Userl 
Client, the CTServer sets up a voice connection to the phone associated 
10 with User2 via message 206. Upon receiving an Alerting message 207 back 
from the phone, the CTSen/er can notify the Userl Client that the called 
phone is ringing by sending the Userl Client a Ring Notification message 
208. When the called user, User2, answers the phone at 209, the CTServer 
will detect the answer and will notify the calling user that the yoice connec- 
15 tion has been accepted by sending the Userl Client a Call Accepted Notifi- 
cation message 210. Now the PSTN voice session is established between 
the Userl phone and the User2 phone. 

The Userl ITCS Client now proceeds to automatically establish an 
H.323 data connection to the User2 H.323 Client. First, the Userl Client 
20 performs an LDAP lookup on a set of network centric LDAP directories using 
the e-mail address of User2 as the distinguished name in the lookup. The 
Userl Client initiates the search by sending a Search Request message 
211. The LDAP directory server responds with a Search Response mes- 
sage 212 identifying the current IP address for the User2 H.323 Client. 
25 Upon receiving the LDAP Search Response message, the Userl Client 
sends an H.323 call Setup message 213 directly to the User2 H.323 Client 
requesting to establish an H.323 connection. The User2 H.323 Client 
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5 receives the Setup message, and responds with an H.323 Call Proceeding 
message 213 followed by an H.323 Alerting message 215. Once the User2 
Client accepts the H.323 call, an H.323 Connect message 216 is sent back 
to the User1 ITCS Client. User1 has completed a harmonized call to a non- 
ITCS H.323 user resulting in a PSTN voice session between both users as 

10 well as an H.323 compliant data session between the Userl ITCS Client and 
the User2 H.323 client. In the above case, as well as in the ITCS to ITCS 
call case, the data session is associated with the voice call and closely 
integrated with the computer desktop applications and the computer user 
interface. 

15 

Descriotion of ITCS Client Software 

The Integrated Telecommunication Collaboration System client soft- 
ware architecture includes four main components, plus services and sub- 
components, which vary depending on the operating environment for which 

20 the particular client is intended. Figure 3 shows a component diagram. 
Data Agent 305 is responsible for establishing a data call to another client 
as requested by the Communication Agent 303. Data Agent 303 is also 
responsible for interacting with directory services to perform address trans- 
lation such as the e-mail to IP address translation which may be required as 

25 part of the call connection message flow previously discussed. In the 
preferred embodiment H.323 is used as the data agent, but any type of data 
agent can be used. Voice Client Agent 304 is responsible for communicat- 
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5 ing directly with the computer telephony server to perform the telephony 
functions. In effect, the data agent handles the data call aspects of the 
ITCS and the voice agent handles the PSTN voice call aspects. 

Communication Agent 303 is responsible for synchronizing the Data 
Agent and the Voice Client Agent. In the preferred embodiment. Communi- 
10 cation Agent 303 makes available an object linking and embedding (OLE) 
compliant interface allowing other applications to make communication 
requests such as place a call, release a call, share an application, collabo- 
rate on an application, etc. The Communication Agent 303 interfaces and 
registers with the Data Agent 305 and the Voice Client Agent 304 and 
1 5 monitors them for events. 

Integration Agent 302 is the component responsible for integrating 
the ITCS communication capability directly into a specific set of computer 
desktop applications as well as integrating the ITCS directly into an operat- 
ing system interface. How the capabilities are integrated into the operating 
20 system interface depends on what operating system the ITCS is designed to 
work with. If the ITCS client software is designed for use with a recent 
version of the Windows™ operating system from Microsoft Corporation such 
as Windows 95™. the ITCS can interface to the system application tray. 
Integration Agent 302 processes user input and makes the appropriate 
25 communication requests of the Communication Agent 303. 

Additional communication services such as conferencing and voice 
mail can be plugged into the Communication Agent to observe communica- 

-14- 
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5 tion events occurring within the Communication Agent, make communication 
requests of the Communication Agent, and present additional graphical 
user interface extensions to the Integration Agent. In most cases, a Suite 
Specific Integration Agent, which serves as a subagent to the Integration 
Agent, will be required to provide direct interface into desktop applications. 
10 The Suite Specific Integration Agent is shown in Figure 3 for completeness. 

Each component shown in Figure 3 can act as an observer and can 
monitor other components events. Likewise, each component may be 
observed by other components. Each component in the architecture ob- 
serves the event space of the component directly underneath it while at the 
15 same time being observed by those components above it. To further illus- 
trate the detailed relationships between the various components of the client 
software, Figures 4. 5. and 6 show examples of the detailed messages 
being passed between the various components. In these examples, it is 
assumed that the data agent is an H.323 compliant data agent and that data 
20 communications take place over a TCP/IP network such as the Internet, 
although the invention can be used with any data communication platform 
and with any type of data network. 

Figure 4 describes the interaction between the client components as 
a user requests a call and sharing of applications, where the user initiates 
25 the request through interacting with the computer. The application is al- 
ready running. Once a request to make a call 401 is received via the user 
interface, the Integration Agent interacts with a Contact Manager to obtain a 
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5 list of contacts at 402. The list is used to select the contact which the user 
wants to establish a call with. 

After the contact information is entered, the Integration Agent re- 
quests the Communication Agent to make a call to the selected person at 
403. The Communication Agent obtains the Directory Number (DN) infor- 

10 mation from the contact information and requests the Voice Client Agent to 
make a PSTN connection with the specified DN at 404. The Voice Client 
Agent interacts with the CTServer to initiate the PSTN connection as speci- 
fied in the previous call flows, as shown at 405, 406, and 407. 

When the CTServer has routed the call to the selected called user 

15 and detects that the called user's phone is ringing, then the Voice Client 
Agent will receive a Ring Notify message 408 which is observed by the 
Communication Agent at 409 and then by the Integration Agent at 410. In 
the preferred embodiment, the Integration Agent updates the appropriate 
on-screen information to notify the user that the phone is ringing. 

20 Eventually the called user will answer the phone. The CTServer de- 

tects the answer and notifies the Voice Client Agent by sending it a Call 
Accepted Notify message 411. The Voice Client Agent posts a Voice Call 
Accepted event 412 to the Communication Agent where it is observed by 
the integration Agent at 413. The Integration Agent updates the appropriate 

25 on screen information to notify the user that the other user has answered the 
call. The Communication Agent then proceeds to request the H.323 Data 
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5 Agent at 414 to make an H.323 data connection to the client specified by the 
e-mail address which was obtained from the contact information. 

The H.323 Data Agent obtains the IP Address from an LDAP direc- 
tory lookup at 415 and 416 and proceeds to setup an H.323 call at 417 and 
418 to the H.323 client specified at the IP address. Once the H.323 data 
10 call is answered, an H.323 Connect message 418 is received by the Data 
Agent. A Data Call Accepted event 419 is posted on the Communication 
Agent where the Integration agent obsen/es it updates on-screen informa- 
tion accordingly at 419 and 420. Now the User is aware that a data session 
is also active and that application sharing may now commence. 
15 At 421 the user requests that an application be shared. A Share Ap- 

plication message 422 is sent by the Integration agent to the Communica- 
tion Agent. This message is replicated and sent as another Share 
Application message 423 from the Communication Agent to the H.323 Data 
Agent. 

20 The harmonized call is terminated if and when the Voice Client Agent 

receives a Release Notify message 424. This message is replicated and 
fonwarded to the Communication Agent as another Release Notify message 
425. The Communication Agent then releases the voice call and sends a 
Voice Call Released message 426 to the Integration Agent, sends a Re- 

25 lease Call message 427 to the H.323 Data Agent and notifies the Integration 
Agent that the data call has been released by sending a Data Call Released 
Message 426. 
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5 Figure 5 describes the interaction between the client components as 

a user requests a call, where the user initiates the request by picking up the 
telephone. The CT Server is made aware of the fact that the user has lifted 
the handset by one of three methods: 1) by being the same server to service 
all telephony requests from that user, for example, because It is a central 

10 office switch or a PBX; 2) by being notified by an AIN or signaling system 7 
(SS7) network; or 3) by having the user dial directly into the CTServer. The 
CTServer will respond with an Off-Hook Notify message 501 to the Voice 
Client Agent within the Client. The Voice Client Agent raises an Outgoing 
Call Requested event 502 to the Communication Agent and the event is 

15 observed by the Integration Agent at 503. The Voice Client Agent requests 
the CTServer to apply dial tone to the phone and collect digits by sending 
the CTSen/er a Collect Digits Message 504. The CTSen/er applies dial tone 
to the PSTN phone and collects the digits entered by the user. Once the 
digits have been collected by the CTServer, the server sends a Digits Re- 

20 ceived Notify message 505 to the Voice Client Agent notifying the Voice 
Client that digit collection has completed and what the digits are. The Voice 
Client Agent responds to the digit collection by sending an Offer Call Re- 
quest message 506 to the CTServer requesting that the server offer a call to 
the user whose DN is specified by the digits. 
25 The CTServer originates the call to the called user and notifies the 

Voice Client Agent when the called user answers at 507. The Communica- 
tion Agent observes the Call Accepted event 508 and translates it to a Voice 
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5 Call Accepted event 509 which is observed by the Integration Agent. The 
Integration Agent updates the appropriate on-screen information. The 
Communication Agent interacts with the Contact Manager at 510 to obtain 
the contact information for the person whose DN matches the DN which was 
entered via digit collection. Once the contact infomiation is retrieved from 
10 the Contact Manager, the Communication Agent requests the H.323 Data 
Agent to make an H.323 connection to the e-mail address which was con- 
tained within the contact information sent at 51 1 . 

The Data Agent establishes the H.323 data connection as specified in 
the earlier section through messages 512. 513 and 514. Once the H.323 
15 connect message 515 is received, the Data Agent sends a Call accepted 
event 516 to the Communication Agent which, in turn, notifies the Integration 
Agent that the Data Call has been accepted at 517. The user then starts an 
application which has been integrated with the ITCS at 518. At 519 and 
520, a GetCallStatus message is exchanged to determine if a user is al- 
20 ready involved in a call. This message is exchanged when a new applica- 
tion which has been integrated with the ITCS is started. It was not 
exchanged in the process illustrated in Figure 4 because it was assumed the 
application was already running. The rest of the process involves applica- 
tion sharing and is identical to that described in Figure 4 and it's accompa- 
25 nying description. Messages 521, 522, and 523 correspond to messages 
421, 422 and 423 in Figure 4. The call release process is not shown in 
Figure 5 for the sake of simplicity. 
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5 Figure 6 shows the interactions between the Harmony client connpo- 

nents as the ITCS processes inconning calls. The CTServer notifies the 
Voice Client Agent that a call is being offered by sending a Call Offered 
message 601, The Voice Client Agent notifies the Communication Agent a 
602 which then notifies the Integration Agent that a Voice Call has been 
10 offered at 603. The Integration Agent displays appropriate caller id informa- 
' tion based on information received from the Contact Manager. 

The Voice Client Agent sends a Receive Notification message 604 to 
the CTServer requesting that it ring the ITCS user's phone. Once the user 
picks up the handset, the CTServer detects it and sends an Off-Hook Notifi- 
15 cation message 605 to the Voice Client Agent notifying the Voice Client 
Agent that the user has just picked up the phone handset. The Voice Client 
Agent sends an Accept Call Request message 606 to the CTServer asking 
that the CTServer connect the user's phone to the incoming call. The Voice 
Client Agent then notifies the Communication Agent that the voice connec- 
20 tion has been accepted at 607. The Communication Agent notifies the 
Integration Agent that the incoming voice call has been accepted at 608 so 
that the appropriate screen information can be updated. 

Shortly following the acceptance of the voice call, the H.323 Data 
Agent receives an H.323 Call Setup message 609 requesting the establish- 
25 ment of a data session. The H.323 Data Agent responds with the appropri- 
ate handshaking. This handshaking is specified in the ITU H.323 
specification. The Data Agent eventually accepts the call by sending an 
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5 H.323 Connect message 610, The Data Agent notifies the Communication 
Agent that the data connection has been accepted at 611. The Integration 
Agent is then informed that the data call has been accepted. Application 
sharing can begin at that point. The same messages are exchanged as 
before, messages 618 through 623 corresponding to messages 518 through 

10 523 in Figure 5. Again, the call release process is not shown in Figure 6 for 
the sake of simplicity. 

Description of ITCS Operating Environment 

Figure 7 illustrates two of many possible network environments in 
15 which the present invention may be used. On the right side of Figure 7, is 
shown User2, 701, whose telephone is interfaced directly to a public 
switched telephone network (PSTN) 705. Additionally User2 has a personal 
computer workstation which is interfaced to both the PSTN and a point-of- 
presence 702 for an Internet service provider (ISP) via a conventional 

20 modem 706. The computer telephony sen/er (CTServer) 704 is connected 
between the PSTN an the ISP point-of-presence. In the preferred embodi- 
ment the server is connected to the PSTN via a primary rate interface (PRI), 
which is a well-known part of the integrated services digital network (ISDN) 
standard. The connection to be ISP point-of-presence is made via either a 

25 T1, T3, or TCP/IP interface. The desktop computer at the User2 location is 
running the ITCS client software as previously discussed. 
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5 The left side of Figure 7 is similar to the right side. However, Userl, 

710 shown on the left side of Figure 7 is using a digital subscriber loop 
(DSL) to interface the desktop computer to the PSTN and the ISP. The 
digital subscriber loop can also be used to interfaced telephones into the 
networks. DSL modem 709 provides the interface point for both the desktop 
10 computer and a DSL telephone. In addition Userl 710 has available a 
conventional telephone that directly interfaces to PSTN 707. CTSen/er 708 
works essentially the same as CTServer 704. According to Figure 7. Userl 
is served by one local exchange carrier (LEG), and User2 is served by a 
different LEG. The two PSTN's are connected via PSTN trunks. Addition- 
15 ally, the two Internet service providers 702 and 711 are connected via the 
public data network 703. In the preferred embodiment the public data 
network is the Internet. 

it should be noted that although Figure 7 shows one example a net- 
work environment in which the invention can operate, there are many other 
20 alternatives. Indeed the public networks could be replaced by privately 
owned networks. Is also possible to interface the GTServer to the ITCS via 
completely different paths depending on what types of networks are avail- 
able and what type of interface the GTServer implements. The client soft- 
ware which implements the invention can operate with standard GTServer 
25 interfaces including the telephony application programming interfaces 
published by Microsoft and Novell. However, one of ordinary skill in the art 
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5 can develop a custom or proprietary interface for the CTServer which would 
work with the present invention. 

Figure 8 illustrates a personal computer workstation on which the cli- 
ent software of the present invention can be operated. Input output (I/O) 
devices such as keyboard 802, mouse 803, and display 804. The display is 
10 used to show an operator a computer desktop on which various information 
is displayed. Such information may include information about the status of 
harmonized calls made with the present invention. System unit 801 is 
connected to all of the I/O devices, and contains memory, media devices, 
and a central processing unit (CPU) all of which together execute the client 
15 software of the present invention and cause the various agents and software 
elements of the present invention to operate when the invention is in use. A 
network interface is normally implemented via an adapter card however for 
the sake of simplicity the shown graphically as interface 805. 

As previously mentioned, appropriate computer program code in 
20 combination with appropriate hardware implements most of the elements of 
the present invention. This computer program code is often stored on 
storage media. This media can be a diskette, hard disk, CD-ROM, or tape. 
The media can also be a memory storage device or collection of memory 
storage devices such as read-only memory (ROM) or random access mem- 
25 ory (RAM). Additionally, the computer code can be transferred to the work- 
station over the Internet or some other type of network. Figure 9 illustrates 
one example of a media. Figure 9 shows a diskette of the type where 
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5 magnetic media 902 is enclosed in a protective jacket 901. Magnetic field 
changes over the surface of the magnetic media 902 are used to encode the 
computer program code. In this way the computer program code is stored 
for later retrieval. 

We have described specific embodiments of our invention which pro- 
10 vides a way in which applications, data communications, and public 
switched telephone network voice calls can be integrated together by a 
desktop computer. One of ordinary skill in the networking and computing 
arts will quickly recognize that the invention has other applications in other 
environments. In fact, many embodiments and implementations are possi- 
15 ble. The following claims are in no way intended to limit the scope of the 
invention to the specific embodiments described. 
We claim: 
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CLAIMS 

1. A method of establishing a harmonized call, the method compris- 
ing the steps of: 

upon receiving a calling user request via computer system 
user interface, requesting a server to establish a telephone connec- 
tion between the server and a calling user; 

receiving a notification that telephone connection between the 
server and the calling user has been established; 

requesting the sender to offer the harmonized call to a called 

user; 

receiving a notification the harmonized call has been accepted 
by the called user; and 

establishing a telephone session and an associated data ses- 
sion with the called user. 
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2. The method of claim 1 further comprising the step of receiving a 
ring notification message while the harmonized call is being of- 
fered to the called user. 

3. The method of claim 1 further comprising the steps of: 

receiving a release notification message; and 

releasing the telephone session and the associated data ses- 
sion with the called user. 

4. The method of claim 2 further comprising the steps of: 

receiving a release notification message; and 

releasing the telephone session and the associated data ses- 
sion with the called user. 

5. The method according to any of claims 1 through 4 wherein the 
associated data session includes application sharing. 
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6. A method of establishing a harmonized call, the method compris- 
ing the steps of: 

10 upon receiving an off-hook notification message, requesting a 

server to establish a telephone connection between the server and a 
calling user; 

receiving a notification that telephone connection between the 
15 server and the calling user has been established; 

requesting the server to offer the harmonized call to a called 

user; 

20 receiving a notification the harmonized call has been accepted 

by the called user; and 

establishing a telephone session and an associated data ses- 
sion with the called user. 

25 
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5 7. The method of claim 6 further comprising the step of receiving a 

ring notification message while the harmonized call is being of- 
fered to the called user. 



10 8. The method of claim 6 further comprising the steps of: 

receiving a release notification message; and 

releasing the telephone session and the associated data ses 
15 sion with the called user. 



9. The method of claim 7 further comprising the steps of: 

20 receiving a release notification message; and 

releasing the telephone session and the associated data ses- 
sion with the called user. 

25 

10. The method according to any of claims 6 through 9 wherein the 
associated data session includes application sharing. 

-28- 



PCTAJS99/08236 



11. A method of receiving a harmonized call, the method comprising 
the steps of: 

receiving a call offered notification message from a server; 

requesting the server to establish a telephone connection be- 
tween the server and a called user; 

receiving a notification that the telephone connection between 
the server and the called user has been established; and 

establishing a telephone session and an associated data ses- 
sion with a calling user integrated telecommunication collaboration 
system. 

12. The method of claim 11 wherein the associated data session in- 
cludes application sharing. 
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5 13. A computer program product for establishing a harmonized call, 

the computer program product having a medium with a computer 
program embodied thereon, the computer program comprising: 

computer program code for receiving a calling user request via 
10 computer system user interface and requesting a server to establish a 

telephone connection between the server and a calling user; 

* computer program code for receiving a notification that the 
telephone connection between the server and the calling user has 
15 been established; 

computer program code for requesting the server to offer the 
harmonized call to a called user; 

20 computer program code for receiving a notification the harmo- 

nized call has been accepted by the called user; and 

computer program code for establishing a telephone session 
and an associated data session with the called user. 

25 
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14. The computer program product of claim 13 wherein the computer 
program further comprises computer program code for receiving a 
ring notification message while the harmonized call is being of- 
fered to the called user. 

15. The computer program product of claim 13 wherein the computer 
program further comprises: 

computer program code for receiving a release notification 
message; and 

computer program code for releasing the telephone session 
and the associated data session with the called user. 

16. The computer program product of claim 14 wherein the computer 
program further comprises: 

computer program code for receiving a release notification 
message; and 
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25 



computer program code for releasing the telephone session 
and the associated data session with the called user. 



17. The computer program product according to any of claims 13 
10 through 16 wherein the associated data session includes applica- 

tion sharing. 



18. A computer program product for establishing a harmonized call, 
15 the computer program product having a medium with a computer 

program embodied thereon, the computer program comprising: 

computer program code for receiving an off-hook notification 
message and requesting a server to establish a telephone connection 
20 between the server and a calling user; 

computer program code for receiving a notification that tele- 
phone connection between the server and the calling user has been 
established; 



computer progranri code for requesting the server to offer the 
harmonized call to a called user; 
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computer program code for receiving a notification the harmo- 
nized call has been accepted by the called user; and 

computer program code for establishing a telephone session 
10 and an associated data session with the called user 



19. The computer program product of claim 18 wherein the computer 
program further comprises computer program code for receiving a 
15 ring notification message while the harmonized call is being of- 

fered to the called user. 



20. The computer program product of claim 18 wherein the computer 
20 program further comprises: 

computer program code for receiving a release notification 
message; and 

25 computer program code for releasing the telephone session 

and the associated data session with the called user. 
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5 

21. The computer program product of claim 19 wherein the computer 
program further comprises: 

computer program code for receiving a release notification 
10 message; and 

computer program code for releasing the telephone session 
and the associated data session with the called user. 



15 



22. The computer program product according to any of claims 18 
through 21 wherein the associated data session includes applica- 
tion sharing. 

20 

23. A computer program product for receiving a harmonized call, the 
computer program product having a medium with a computer pro- 
gram embodied thereon, the computer program comprising: 

25 computer program code for receiving a call offered notification 

message from a server; 
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5 computer program code for requesting the server to establish a 

telephone connection between the server and a called user; 

computer program code for receiving a notification that the 
telephone connection between the server and the called user has 
10 been established; and 

computer program code for establishing a telephone session 
and an associated data session. 



15 



24. The computer program product of claim 23 wherein the associ- 
ated data session includes application sharing. 



20 25. An integrated telecommunication collaboration system client 

comprising: 

a voice agent for establishing telephone sessions through a 

server; 

25 

a data agent for establishing associated data sessions with 
other clients; 
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a communications agent connected to the voice agent and the 
data agent for synchronizing and observing the voice agent and the 
data agent, the communications agent having interfaces for a plurality 
of service agents; and 

an integration agent connected to the communications agent 
for processing user input, making requests of the communications 
agent, the integration agent having interfaces for a plurality of appli- 
cations. 

26. A computer program product for causing a computer to imple- 
ment an integrated telecommunication collaboration system client, 
the computer program product including a medium with a com- 
20 puter program embodied thereon, the computer program com- 

prising: 

computer program code for establishing a voice agent for es- 
tablishing telephone sessions through a server; 

25 

computer program code for establishing a data agent for es- 
tablishing associated data sessions with other clients; 
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5 

computer program code for establishing a communications 
agent connected to the voice agent and the data agent for synchro- 
nizing and observing the voice agent and the data agent, the com- 
munications agent having interfaces for a plurality of service agents; 
10 and 

computer program code for establishing an integration agent 
connected to the communications agent for processing user input, 
making requests of the communications agent, the integration agent 
15 having interfaces for a plurality of applications. 

27. A computer system including an integrated telecommunication 
collaboration client, the computer system comprising: 

20 

a plurality of input/output devices; 

a network connection; 

25 a system unit disposed between the plurality of input/output 

devices and the network connection, the system unit including: 
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5 a voice agent for establishing telephone sessions through a 

server; 

a data agent for establishing associated data sessions with 
other clients; 

10 

a communications agent connected to the voice agent and the 
data agent for synchronizing and observing the voice agent and the 
data agent, the communications agent having interfaces for a plurality 
of service agents; and 

15 

an integration agent connected to the communications agent 
for processing user input, making requests of the communications 
agent, the integration agent having interfaces for a plurality of appli- 
cations. 

20 

28. An integrated telecommunication collaboration system client 
comprising: 

25 means for establishing a telephone session through a server; 
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5 means for establishing a data session with another client, 

wherein the data session is associated with the telephone connec- 
tion; 

communications means connected to the telephone connec- 
10 tion means and the multimedia data connection means for synchro- 

nizing and observing the telephone connection means and the 
multimedia data connection means, the communications means hav- 
ing interfaces for a plurality of service agents; and 

15 integration means connected to the communications means for 

processing user input, making requests of the communications 
means, the integration means having interfaces for a plurality of ap- 
plications. 



20 



25 



29. An integrated telecommunication collaboration network compris- 
ing: 

a public switched telephone network (PSTN); 

a public data network having a service provider point-of- 
presence for providing connectivity to the public data network; 

-39- 



wo 99/55099 



PCT/US99/08236 



a computer telephony server disposed between the PSTN and 
the public data network; 

a user telephone connected to the PSTN; and 

10 

an integrated telecommunication collaboration system con- 
nected to the PSTN and the public data network, the integrated tele- 
comnnunication collaboration system including a communications 
agent for synchronizing and observing a voice agent and a data 
15 agent, the communications agent having interfaces for a plurality of 

service agents, the system also including an integration agent con- 
nected to the communications agent for processing user input, mak- 
ing requests of the communications agent, the integration agent 
having interfaces for a plurality of applications. 

20 

30. An integrated telecommunication collaboration network compris- 
ing: 

25 a public switched telephone network (PSTN); 
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a public data network having a service provider point-bf- 
presence for providing connectivity to the public data network; 

a computer telephony server disposed between the PSTN and 
the public data network; 



10 



a user telephone connected to the PSTN; and 

an integrated telecommunication collaboration means con- 
nected to the PSTN and the public data network. 



15 
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FIG. 3 
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